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DETAILED ACTION 

Specification 

The disclosure is objected to because it contains an embedded hyperlink and/or 
other form of browser-executable code. Applicant is required to delete the embedded 
hyperlink and/or other form of browser-executable code. See MPEP § -608.01 . There 
are a number of URLs in the specification that must be removed. URLs are found on 
pages 15 and 28. 

The disclosure is objected to because of the following informalities: the 
Appendices A, B, and C on pages 36-56 are improper because they are printouts from 
web pages that may be copyrighted. They include figures and graphic "buttons" icons, 
URLs etc. The appendices therefore must be removed and the specification may be 
amended to include the text of the appendices. Appropriate correction is required. 

Claim Rejections - 35 USC § 102 
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on safe in this country, more than one year prior to the date of application for patent in the United 
States. 
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Claims 1-1 1, 26-37, 41-44, and 46-56 are rejected under 35 U.S.C. 102(b) as 
being anticipated by U.S. Patent 5,963,675 to van der Wal et al. 

With regard to claim 1, van der Wal discloses a method for stabilizing motion in a 
sequence of frames (column 6, lines 16-41). 

Van der Wal further discloses identifying one or more features in a first frame in 
the sequence of frames (Fig. 2a and column 6, lines 31-41). The V in the frames is 
considered to be the feature in the sequence of frames. 

Van der Wal further discloses calculating tracked positions for one or more 
features in each other frame in the sequence of frames based on the features in the first 
frame (Fig. 2b and column 6, lines 42-46); 

Van der Wal further discloses calculating ideal positions for the features in each 
other frame in the sequence of frames based on the tracked positions (Fig. 2c and 
column 6. lines 45-52); 

Van der Wal further discloses identifying transformation information based on the 
tracked positions and the calculated positions (Fig. 2c and column 6, lines 45-52); and 

Van der Wal further discloses transforming each other frame in the sequence of 
frames by adjusting pixels in each other frame based on the transformation information 
(Figs. 2c, 2d and 3 and column 6, lines 50-64). 



With regard to claim 2, van der Wal discloses the method of claim 1 , further 
comprising determining whether the sequence of frames was recorded with fast camera 
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motion or slow camera motion (column 6, lines 25-30). Van der Wal discloses that 
several modes are known in the art for stabilizing video images that require determining 
the type of camera motion. 

With regard to claim 3, van der Wal discloses the method of claim 2, further 
comprising when it is determined whether the sequence of frames was recorded with 
fast camera motion, performing matching to calculate tracked position for each feature 
in a second frame in the sequence of frames (column 6, lines 25-30 and 37-53, and 
column 7, lines 20-40 and Figs. 2a-2d and 4). Van der Wal discloses performing a 
hierarchical correlation, which is interpreted as matching between images. 

With regard to claim 4, van der Wal discloses the method of claim 3, wherein 
matching further comprises identifying one or more features in the second frame and 
matching the one or more features in the first frame to the one or more features in the 
second frame (column 6, lines 25-30 and 37-53, and column 7, lines 20-40 and Figs. 
2a-2d and 4). 

With regard to claim 5, Wal discloses the method of claim 2, further comprising 
when it is determined that the sequence of frames was recorded with slow camera 
motion, performing global tracking to calculate estimated positions for each of the one 
or more features in the second frame (column 6, lines 25-30 and 37-53 and Figs. 2a- 
2d). 
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With regard to claim 6, Wal discloses the method of claim 5, wherein global 
tracking further comprises selecting a first feature and searching a portion of the second 
frame to find a feature that corresponds to the selected first feature in the first frame 
(column 6, lines 64-67 and column 7, lines 1-40). Wal discloses a method of 
determining correlation between the frames in a pyramid hierarchy scheme to determine 
the amount of motion between the two frames. The method finds correlations for small 
neighborhoods of pixels or areas of interest and works its way up. 

With regard to claim 7, Wal discloses the method of claim 1, wherein calculating 
tracked positions further comprises performing local tracking (column 6, lines 37-53 and 
Figs. 2a-2d). The process of tracking the object of interest from frame to frame is 
interpreted as local tracking. 

With regard to claim 8, Wal discloses the method of claim 1 , wherein calculating 
ideal positions further comprises plotting the tracked positions in an X, Y coordinate 
graph; and drawing a motion trajectory by connecting a first one of the plotted tracked 
positions with a last one of the plotted tracked positions (Figs. 2a-2d). Wal discloses 
the process of plotting the points of interest in several consecutive frames. The motion 
trajectory is plotted in the figures and are considered to possess X, Y coordinate points. 
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With regard to claim 9, Wal discloses the method of claim 8, wherein the motion 
trajectory is linear (Fig. 2d). 

With regard to claim 10, Wal discloses the method of claim 8, wherein the motion 
trajectory is non-linear (Fig. 2b and 2c). 

With regard to claim 1 1 , Wal discloses the method of claim 8, wherein the ideal 
comprises positions on the motion trajectory (Fig. 2d). 

With regard to claim 26, Van der Wal discloses a system comprising a computer 
including a processor and a memory (Figs. 5 and 6). 

Van der Wal further discloses a sequence of frames stored in the memory (Figs. 
2a-2d). 

Van der Wal further discloses a program stored in memory of the computer, 
wherein the program is executed by the processor of the computer to identify one or 
more features in a first frame in the sequence of frames (Fig. 2a and column 6, lines 31- 
41). The V in the frames is considered to be the feature in the sequence of frames. 

Van der wal further discloses calculating tracked positions for one or more 
features in each other frame in the sequence of frames based on the features in the first 
frame (Fig. 2b and column 6, lines 42-46). 
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Van der Wal further discloses calculating ideal positions for the features in each 
other frame in the sequence of frames based on the tracked positions (Fig. 2c and 
column 6, lines 45*52). 

Van der Wal further discloses identifying transformation information based on the 
tracked positions and calculated positions (Fig. 2c and column 6, lines 45-52). 

Transform each other frame in the sequence of frames by adjusting pixels in 
each other frame based on the transformation information (Figs. 2c, 2d and 3 and 
column 6, lines 50-64). 

With regard to claim 27, Van der Wal discloses the system of claim 26, further 
comprising: a video camera for recording a sequence of frames (column 6, lines 25-30). 

With regard to claim 28, Van der Wal discloses the system of claim 27, wherein 
execution of the program further comprises determining whether the sequence of 
frames was recorded with fast camera motion or slow camera motion (column 6, lines 
16-30). 

With regard to claim 29, Van der Wal discloses the system of claim 28, wherein 
execution of the program further comprises when it is determined that the sequence of 
frames was recorded with fast camera motion, performing matching to calculate tracked 
positions for each feature in a second frame in the sequence of frames (column 6, lines 
25-30 and 37-53, and column 7, lines 20-40 and Figs. 2a-2d and 4). Van der Wal 
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discloses performing a hierarchical correlation, which is interpreted as matching 
between images. 

With regard to claim 30, Van der Wal discloses the system of claim 29, wherein 
the execution of the program further comprises identifying one or more features in the 
second frame and matching the one or more features in the first frame to the one or 
more features in the second frame (column 6, lines 16-52 and Figs. 2a-2d). 

With regard to claim 31 , Van der Wal discloses the system of claim 28, wherein 
execution of the program further comprises when it is determined that the sequence of 
frames was recorded with slow camera motion, performing global tracking to calculate 
estimated positions for each of the one or more features in the second frame (column 6, 
lines 16-52 and Figs. 2a-2d). Van der Wal discloses that a determination is made as to 
remove motion depending on the kind of camera motion and then the process of 
matching points of interest in the sequence of frames is explained. 

With regard to claim 32, Van der Wal discloses the system of claim 31 , wherein 
execution of the program further comprises selecting a first feature in the first frame and 
searching a portion of the second frame to find a feature that corresponds to the 
selected first feature in the first frame (column 6, lines 64-67 and column 7, lines 1-40). 
Wal discloses a method of determining correlation between the frames in a pyramid 
hierarchy scheme to determine the amount of motion between the two frames. The 
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method finds correlations for small neighborhoods of pixels or areas of interest and 
works its way up. 

With regard to claim 33, Van der Wal discloses the system of claim 27, wherein 
execution of the program further comprises plotting the tracked positions in an X, Y 
coordinate graph (Figs. 2a-2b and column 6, lines 30-41), and drawing a motion 
trajectory by connecting a first one of the plotted tracked positions with a last one of the 
plotted tracked positions (Figs. 2a-2b and column 6, lines 42-53). Van der Wal 
discloses that the motion of the point of interest is tracked from one frame to the next to 
determine displacement. This is interpreted as a motion trajectory vector between 
plotted tracked positions. 

With regard to claim 34, Van der Wal discloses the system of claim 33, wherein 
the motion trajectory is linear (Fig. 2d). 

With regard to claim 35, Van der Wal discloses wherein the motion trajectory is 
non-linear (Fig. 2b). 

With regard to claim 36, Van der Wal discloses wherein the ideal positions 
comprise positions on the motion trajectory (Figs. 2c and 2d). 
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With regard to claim 41 , the discussion of claims 26 and 27 applies. Van der Wal 
discloses a camera for use in the system discussed (column 6, lines 25-30). 

With regard to claim 42, the discussion of claims 28 and 29 applies. 

With regard to claim 43, the discussion of claims 28 and 29 applies. Van der Wal 
discloses that a determination is made on how to sense camera motion as either fast or 
slow in order to track objects of interest column 6, lines 20-30). 

With regard to claim 44, Van der Wal discloses the system of claim 41 , further 
comprising means for calculating estimated positions for features in a frame (column 6, 
lines 1 6-52 and Figs. 2a-2d). 

Van der Wal further discloses means for calculating tracked positions based on 
the estimated positions (Figs. 2a-2c) and means for drawing a motion trajectory based 
on the tracked positions (Fig. 2d). 

With regard to claim 46, the discussion of claim 33 applies. 

With regard to claim 47, Van der Wal discloses wherein the ideal positions 
comprise positions on the motion trajectory (Figs. 2a-2d). 
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With regard to claim 48, the discussion of claim 1 applies. Van der Wal discloses 
a computer readable storage medium in use with the method disclosed in claim 1 (Figs. 
5 and 6). 

With regard to claims 49-55, the discussions of claims 2-8 apply respectively. 
With regard to claim 56, the discussion of claim 1 1 applies. 

Claim Rejections • 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 12, 20-21 , 24-25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over U.S. Patent 5,963,675 to van der Wal et al. 

With regard to claim 12, Wal discloses the method of claim 1 1 , wherein an ideal 
position for a point of interest has X and Y coordinates (Figs. 2a-2d), but does not 
explicitly disclose where a point of interest has a same X-coordinate and a different Y- 
coordinate as compared to a tracked position for the point of interest. It is considered 
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obvious that the point of interest can have any tracked position relative to the ideal 
position. For example it appears in Fig. 2d that the ideal position for the pixel of interest 
has the same Y, but different X coordinates as the tracked positions. Therefore it would 
have been obvious to one of ordinary skill in the art at the time of invention to enable 
tracking of coordinates where the point of interest has the same X-coordinates and 
different Y-coordinates with respect to the tracked positions in order to track any kind of 
motion between the frames. 

With regard to claim 20, van der Wal discloses a method for stabilizing a 
sequence comprising calculating a first position of a point of interest in a first frame (Fig. 
2a and column 6, lines 31-41). The V in the frames is considered to be the first 
position of a point of interest in frame 200 at F(t). 

Van der Wal further discloses identifying estimated positions of points of interest 
in a second frame and a third frame that correspond to the point of interest in the first 
frame (Fig. 2a and column 6, lines 31-41). The V in the frames is considered to be the 
point of interest displayed in the sequence of frames. 

Van der Wal further discloses identifying tracked positions of points of interest in 
the second frame and third frame based on the estimated positions of the point of 
interest, wherein the tracked positions comprise a second position for the point of 
interest in the second frame and a third position for the point of interest in the third 
frame (Fig. 2a and column 6, lines 31-41). The V in the frames is considered to be the 
feature in the second and third frames shown in elements 200 at F(t+1) and F(t+2). 
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Van der Wal further comprises plotting the first position, the second position, and 
the third position on an X, Y coordinate graph (Figs. 2a-2b and column 6, lines 42-53). 

Van der Wal does not explicitly disclose connecting the first position to the third 
position to the third position on the X, Y coordinate graph, wherein ideal positions of the 
point of interest in the first frame, second frame, and third frame lie on the connection. 
However Van der Wal discloses plotting the points of interest on a common X, Y 
coordinate plain and using that plot to determine image shift and other stabilization 
information and the plots are connected in some form to determine movement from one 
frame to the next (Figs. 2a-2d and column 6, lines 42-53). The order in which the points 
are connected could obviously vary according to the stabilization effect desired. 
Therefore it would have been obvious to one of ordinary skill in the art at the time of 
invention to connect or use the spatial relationship between points in the plot of 
sequential frames as taught by Van der Wal in any order to achieve a desired 
stabilization effect. 

With regard to claim 24, Van der Wal discloses determining whether to perform 
global tracking or matching to generate estimated points of interest (Figs. 2a-2d and 
column 6, lines 42-53 and Fig.4 and column 7, lines 10-40). Van der Wal discloses how 
a hierarchy is used to match or correlate the pixel neighborhoods in a pyramid matching 
sequence. 



* 
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With regard to claim 25, Van der Wal discloses wherein the determination is 
based on whether camera motion for the sequence of frames is fast or slow (column 6, 
lines 16-30). Van der Wal discloses that a determination is made between whether the 
motion of the image is caused by fast motion or slow motion. 



Claims 13, 14, 45, and 57-60 are rejected under 35 U.S.C, 103(a) as being 
unpatentable over the combination of U.S. Patent 5,963,675 to van der Wal et al. and 
U.S. Patent 6,459,822 to Hathaway et al. 

With regard to claim 13, van der Wal discloses the method of claim 1 but does 
not explicitly disclose wherein transformation information comprises rotation, scaling, 
shearing, and/or translation information for each pixel of each other frame in the 
sequence of frames. Hathaway discloses a method for stabilizing and registering a 
video image in multiple video fields and compensating for rotation, magnification (or 
scale and translation (column 2, lines 49-62). Hathaway teaches that the magnification 
change, translation and rotation information is used to de-magnify, de-translate, and de- 
rotate the other image frames to give the appearance of the key area being motionless 
(column 1 1 , lines 26-34). Therefore it would have been obvious to one of ordinary skill 
in the art at the item of invention to use the magnification, and rotation operations 
additionally in the translation system of van der Wal in order to give the appearance of 
the key area being motionless or stable. 
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With regard to claim 14, Hathaway discloses wherein transforming each other 
frame further comprises adjusting each pixel in each other frame based on the rotation, 
scaling, shearing, and/or translation information (column 1 1 , lines 26-34). Hathaway 
discloses modifying the data or pixel information in each field other than the key field or 
frame. 

With regard to claim 45, Van der wal discloses the system of claim 41 , but does 
not explicitly disclose wherein transforming information comprises rotation, scaling, 
shearing, and/or translation information for each pixel of a frame. Hathaway discloses a 
method for stabilizing and registering a video image in multiple video fields and 
compensating for rotation, magnification or scale and translation (column 2, lines 49- 
62). Hathaway teaches that the magnification change, translation and rotation 
information is used to de-magnify, de-translate, and de-rotate the other image frames to 
give the appearance of the key area being motionless (column 11, lines 26-34). 
Therefore it would have been obvious to one of ordinary skill in the art at the item of 
invention to use the magnification, and rotation operations additionally in the translation 
system of Van der Wal in order to give the appearance of the key area being motionless 
or stable. 

With regard to claim 57, the discussion of claim 13 applies. 
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With regard to claims 58-60, the discussions of claims 15-17 apply. 

Claims 18, 19, and 40 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the combination of U.S. Patent 5,963,675 to van der Wal et al. and 
U.S. Patent 6,636,220 to Szeliski et al. 

With regard to claim 18, van der Wal discloses the method of claim 1 , but does 
not disclose performing frame averaging. Szeliski discloses performing frame 
averaging and teaches that it is used in order to compensate for ghosting and blurring 
(column 22, lines 57-67 and column 23, lines 1-16). Therefore it would have been 
obvious to one of ordinary skill in the art at the time of the invention to perform frame 
averaging in order to compensate for ghosting and blurring. 

With regard to claim 19, the discussion of claim 18 applies. Szeliski performs de- 
blurring which is equivalent to sharpening. 

With regard to claim 40, the discussions of claims 18 and 19 apply. 



Claims 15, 21 and 37 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over U.S. Patent 5,963,675 to van der Wal et al. hereinafter referred to as Van der Wal 
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5 in view of U.S. Patent 6,567,574 to Van der Wal et al. herein after referred to as Van 
derWal 6. 

With regard to claim 15, Van der Wal 5 discloses the method of claim 1 , but does 
not disclose wherein transforming each other frame in a sequence of frames further 
comprises performing affine transformation using the tracked positions and the ideal 
positions. Van der Wal 6 discloses affine transformation (column 7, lines 20-25). Van 
der Wal 6 teaches that affine transformation may be used as a general transformation in 
aligning frames from video signals in the method of Van der Wal 5. Therefore it would 
have been obvious to one of ordinary skill in the art at the time of invention to use affine 
transformation as taught by Van der Wal 6 in the method of Vander Wal 5 in order to 
perform general transformation in video frames. 

With regard to claim 21, the discussion of claims 15 and 20 apply. 

With regard to claim 37, the discussion of claims 15 and 26 apply. 

Claims 16, 17, 22, 23, 38, and 39 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the combination of U.S. Patent 5,963,675 to van der Wal et al. 
hereinafter referred to as Van der Wal 5 in view of U.S. Patent 6,567,574 to Van der 
Wal et al. herein after referred to as Van der Wal 6 and further in view of U.S. Patent 
6,459,822 to Hathaway et al. 
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With regard to claim 16, van der Wal 5 and Van der Wal 6 disclose the method of 
claim 15, wherein performing affine transformation further comprises determining values 
of coefficients representing rotation, scaling, shearing, and/or translation information 
using the tracked positions and the ideal positions. Hathaway discloses a method for 
stabilizing and registering a video image in multiple video fields and compensating for 
rotation, magnification (or scale and translation (column 2, lines 49-62). Hathaway also 
discloses using coefficients as a measure of correlation, which provides a measure of 
rotation, translation, or magnification (column 6, lines 1-11). Hathaway teaches that the 
magnification change, translation and rotation information is used to de-magnify, de- 
translate, and de-rotate the other image frames to give the appearance of the key area 
being motionless (column 1 1 , lines 26-34). Therefore it would have been obvious to 
one of ordinary skill in the art at the item of invention to use the magnification, and 
rotation operations additionally in the translation system of van der Wal in order to give 
the appearance of the key area being motionless or stable. 

With regard to claim 17, Hathaway discloses applying the values of the 
coefficients to each pixel of the first frame to obtain positions of each pixel in a second 
frame (column 6, lines 1-11). Hathaway discloses that the correlation coefficients are 
applied to determine how to translate or modify the pixels in the other frames. 
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With regard to claim 22, Van der Wal 5 and Van der Wal 6 disclose performing 
affine transformation of claim 21, but does not disclose determining values of 
coefficients representing rotation, scaling, shearing, and/or translation information using 
the tracked information and the ideal positions. Hathaway discloses a method for 
stabilizing and registering a video image in multiple video fields and compensating for 
rotation, magnification (or scale and translation {column 2, lines 49-62). Hathaway also 
discloses using coefficients as a measure of correlation, which provides a measure of 
rotation, translation, or magnification (column 6, lines 1-11). Hathaway teaches that the 
magnification change, translation and rotation information is used to de-magnify, de- 
translate, and de-rotate the other image frames to give the appearance of the key area 
being motionless (column 1 1 , lines 26-34). Therefore it would have been obvious to 
one of ordinary skill in the art at the item of invention to use the magnification, and 
rotation operations additionally in the translation system of van der Wal in order to give 
the appearance of the key area being motionless or stable. 

With regard to claim 23, Hathaway discloses applying the values of coefficients 
to each pixel of the first frame to obtain positions of each pixel in the second frame 
(column 6, lines 1-11). Hathaway discloses that the correlation coefficients are applied 
to determine how to translate or modify the pixels in the other frames. 

With regard to claim 38, Van der Wal 5 and Van derWal 6 disclose the system of 
claim 37, but does not explicitly disclose wherein execution of the program further 



Application/Control Number: 10/003,329 Page 20 

Art Unit: 2623 

comprises determining values of coefficients representing rotation, scaling, shearing, 
and/or translation information using the tracked positions and the ideal positions. 
Hathaway discloses a method for stabilizing and registering a video image in multiple 
video fields and compensating for rotation, magnification or scale and translation 
(column 2, lines 49-62). Hathaway teaches that the magnification change, translation 
and rotation information is used to de-magnify, de-translate, and de-rotate the other 
image frames to give the appearance of the key area being motionless (column 11, 
lines 26-34). Therefore it would have been obvious to one of ordinary skill in the art at 
the item of invention to use the magnification, and rotation operations additionally in the 
translation system of Van der Wal in order to give the appearance of the key area being 
motionless or stable. 

With regard to claim 39, Hathaway discloses wherein the execution of the 
program further comprises applying the values of coefficients to each pixel of the first 
frame to obtain positions of each pixel in a second frame (column 6, lines 1-11). 
Hathaway discloses that the correlation coefficients are applied to determine how to 
translate or modify the pixels in the other frames. 
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Contact Information 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Wes Tucker whose telephone number is 703-305-6700. 
The examiner can normally be reached on 9AM-5PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on (703)308-6604. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 

Wes Tucker 
10-20-2004 




Primary Examiner 



